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Abstract 

Correctly  identifying  tracks  is  a  difficult  but  important  capability  for  US  Navy 
ships  and  aircraft.  It  is  difficult  because  of  the  inherent  uncertainty,  complexity,  and 
short  timelines  involved.  It  is  important  because  the  price  of  failure  is  missed  or  civilian 
engagements  and  fratricide.  Today,  Navy  ships  and  aircraft  primarily  use  an  If-Then  rule 
based  system  evaluating  radar  and  IFF  information  to  perform  Combat  Identification 
(CID).  To  cope  with  the  uncertainty  and  complexity  of  CID,  Bayesian  networks  have 
been  suggested  to  integrate  Radar,  IFF,  and  other  lower  quality  sources  to  perform  the 
Identification  determination.  The  goal  of  this  project  show  that  Bayesian  Networks  can 
be  used  to  support  CID  investment  decisions.  Two  investments,  a  new  sensor  and  good 
maintenance  were  compared  in  a  difficult  CID  scenario  in  four  different  environments. 

1.0  Introduction 

Correctly  identifying  tracks  is  a  difficult  but  important  capability  for  US  Navy 
ships  and  aircraft.  Combat  Identification  (CID)  is  difficult  because  of  the  inherent 
uncertainty,  complexity,  and  short  timelines  involved.  There  is  uncertainty  in  associating 
evidence  to  an  object,  uncertainty  in  association  between  evidence  and  identity, 
classification  and  intention.  CID  is  complex  because  of  the  number  of  objects,  their 
interactions,  the  variety  of  observations  of  these  objects,  and  the  concern  that  an  enemy 
might  try  to  deliberately  to  confuse  or  deceive  your  sensors.  In  many  cases  the  time  to 
decide  on  whether  to  shoot  an  object  and  by  implication  the  time  to  perform  a  combat 
identification  is  very  short. 

CID  is  important  because  the  price  of  failure  is  missed  or  civilian  engagements 
and  fratricide.  In  1994,  the  US  shot  down  two  of  there  US  Army  helicopters  and  killed  26 
people.  [1]  In  1988,  the  USS  Vincennes  shot  down  a  commercial  airliner.  [2]  In  1987, 
two  missiles  fired  from  aircraft  hit  the  USS  Stark  because  they  did  not  shoot  the  aircraft 
or  incoming  missiles.  [3]. 

Today,  Navy  ships  and  aircraft  primarily  use  an  If-Then  rule  based  system 
evaluating  radar  and  IFF  information  to  perform  Combat  Identification  (CID).  To  cope 
with  the  uncertainty  and  complexity  of  CID,  Bayesian  networks  have  been  suggested  to 
integrate  Radar,  IFF,  and  other  lower  quality  information  sources  to  perform  the  CID. 

The  goal  of  this  project  show  that  the  Bayesian  Networks  can  be  used  to  support 
CID  investment  decisions.  Two  investments,  a  new  sensor  and  good  maintenance  were 
compared  in  a  difficult  CID  scenario  in  four  different  environments.  The  usefulness  of 
the  two  investments  was  compared  examining  the: 

•  Separation  in  probability  that  the  object  is  hostile,  neutral,  or  friendly  and 

•  Separation  in  the  utility  between  the  decision  to  shoot  or  not  to  shoot. 

Section  2  of  this  paper  provides  background  information  on  combat  identification, 
decision  analysis,  Bayesian  networks,  and  knowledge  engineering.  Section  3  describes 
the  project  while  section  4  provides  the  results.  Section  5  is  an  evaluation  of  the  results. 
Section  6  summarizes  the  key  results  of  the  paper. 
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2.0  Background 

This  project  applies  the  techniques  from  decision  analysis  and  Bayesian  networks  to 
address  the  challenges  of  combat  identification.  The  CID  network  was  developed  using 
good  knowledge  engineering  practices.  An  introduction  to  these  topics  is  provided  in  this 
section. 

2.1  Combat  Identification 

Combat  Identification  is  the  process  of  assigning  an  identity  and  classification  to  each 
object  detected  by  the  host  platform.  As  shown  in  Figure  (1),  identity  has  seven  possible 
states.  [4]  The  three  most  important  for  this  project  are  hostile,  neutral,  and  friendly. 
Classification  is  more  complex.  It  describes  where  the  object  is  found  -  air,  surface,  etc. 
It  also  describes  the  type  of  object  at  different  levels  of  detail  from  platform  type  -  like 
fighter  -  all  the  way  down  to  each  individual  aircraft.  Nationality  tells  for  which  country 
the  aircraft  flies.  Finally,  mission  or  intent  provides  information  on  what  the  object  is 
doing  such  as  combat  air  patrol  (CAP)  or  strike.  Accurately  determining  this  information 
is  important,  as  it  is  a  key  input  to  the  various  decisions  including: 

•  Whether  to  collect  more  information  about  the  object, 

•  Whether  to  stop  the  object  from  performing  it’s  mission,  and 

•  How  best  to  stop  the  object  from  performing  it’s  mission. 


COMBAT  ID  TAXONOMY 
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Figure  1  Combat  ID  Taxonomy 


Figure  (2)  shows  why  it  is  important  to  perform  CID  correctly  [5].  If  an  object  is 
hostile  and  you  don’t  destroy  it  ships  &  crews  are  lost  and  eventually  wars  are  lost.  But 
if  the  object  is  friendly  or  neutral  and  you  destroy  it  then  lives  are  lost  and  wars  can  be 
started. 
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Algorithms  to  automatically  assign  an  ID  date  back  to  when  the  Navy  started  putting 
computers  in  ships  [6].  The  first  efforts  were  coded  directly  into  the  software  and  were 
hard  to  adapt  to  different  situations.  In  the  1980’s,  rule  based  expert  systems  were 
introduced.  These  systems  were  more  flexible  because  the  rules  could  be  adapted  to 
changing  situations.  These  systems  tended  to  use  only  high  quality  evidence  like 
position,  velocity,  and  the  results  of  Identification  Friend  or  Foe  (IFF)  interrogations. 
Operations  and  exercise  noted  problems  with  these  systems  and  the  US  Navy  has 
thoroughly  explored  the  causes  of  these  problems  and  suggested  solutions.  [4],  [5],  [7], 
[8],  [9],  [10],  [11] 


The  Combat  ID  Functional  Allocation  Working  Group  suggested  an  architecture, 
shown  in  figure  (3)  to  integrate  all  information  including  high  quality  data,  like  radar  and 
IFF,  and  lower  quality  data  like  Electronic  Support  (think  radar  detectors)  and 
intelligence  information.  [8]  In  July  of  2001,  the  Office  of  Naval  Research  released  a 
broad  agency  announcement  (BAA)  for  composite  combat  identification  to  prototype 
systems  that  could  perform  this  data  fusion  shown  in  Figure  (4).  [4]  One  of  the  methods 
suggested  in  the  BAA  was  the  use  of  Bayesian  Networks. 


Page  4  of  13 


Combat  ID  Architecture 
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Figure  3  Combat  ID  Architecture 
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Figure  4  Contributors  to  Combat  ID 
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2.2  Decision  Analysis 

Clemen’s  book  “Making  Hard  Decisions”  [13]  is  an  excellent  introduction  to 
decision  analysis.  Some  of  the  key  ideas  from  this  text  and  the  course  notes  on  decision 
theory  relevant  to  this  project  [12]  are  that  the  interaction  of  decision  options  and  states 
of  the  world  are  consequences.  For  simple  decision  situations  this  interaction  can  be 
recorded  in  a  table  with  options  on  one  axis  and  possible  states  of  the  world  as  the  other. 
Each  cell  in  the  table  can  be  assigned  a  utility  score  with  0  for  the  least  desirable 
consequence  and  1  for  the  most  desirable  consequence.  With  these  two  cells  as  anchors 
all  the  other  cells  can  be  assigned  a  based  on  preference  for  the  consequence  compared  to 
these  best  and  worst  consequence.  From  this  table  an  expected  utility  can  be  calculated 
for  each  option.  The  best  option  is  the  one  that  has  the  maximum  expected  utility.  This 
is  represented  mathematically  in  Equation  (1). 


Action 


N 


Taken  =  arg  max  ^l/(Ck)py 

a-  H 


Equation  1 


2.3  Bayesian  Networks 

Jensen  [14],  defines  a  Bayesian  Network  as  consisting  of  the  following: 

•  A  set  of  variables  and  a  set  of  directed  edges  between  variables 

•  Each  variable  has  a  finite  set  of  mutually  exclusive  states. 

•  The  variables  together  with  the  directed  edges  form  a  directed  acyclic  graph 
(DAG) 

•  To  each  variable  A  with  parents,  Bi,  ...  Bn,  there  is  attached  the  potential  table 

P(AIB, . Bn) 

These  graphical  models  are  effective  and  efficient  method  of  dealing  with  uncertainty. 

An  example  Bayesian  network  is  shown  in  figure  (6).  The  standard  problem  involving  a 
Bayesian  network  is  given  evidence  calculate  the  probability  of  various  states  of 
hypothesis  acting  through  various  mediating  variables.  Bayesian  networks  are  easy  to 
create/  modify.  These  networks  can  mix  historical,  modeling  and  simulation,  and  expert 
judgment.  The  structure  and  parameters  can  learned  from  data.  They  offer  several 
advantages  over  standard  statistical  techniques  because  they  make  use  of  the  conditional 
independence  to  reduce  the  number  of  parameters  to  estimate.  They  are  easy  to  compute. 
Efficient  algorithms  were  developed  in  the  late  1980’s  for  computing  probabilities.  The 
can  accommodate  missing  data.  There  are  fewer  parameters  to  estimate  than  standard 
statistical  model.  These  graphical  models  are  more  understandable  than  neural  nets. 
Algorithms  also  exist  to  calculate  the  most  probable  explanation  and  consistency  of 
evidence. 
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WHAT  IS  A  BAYESIAN  NETWORK? 


Bayesian  Network  Consists  of: 

•Variables  (Evidence,  Mediating,  Hypothesis) 
•States 

•Directed  Connections 
•Conditional  Probabilities 
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Figure  5  Bayesian  Network 


2.4  Knowledge  Engineering 

To  create  effective  Bayesian  networks  good  knowledge  engineering  is  required. 
[15]  Knowledge  engineering  is  the  process  of  eliciting,  modeling  and  evaluating 
knowledge  from  an  expert  so  that  it  can  be  used  to  support  decision-makers. 

The  first  step  in  the  knowledge  elicitation  process  is  to  define  the  goal  of  the 
modeling  process.  The  next  step  is  to  select  possible  nodes  in  the  network.  These  nodes 
will  be  evidence,  hypothesis  or  mitigating  variables.  The  third  step  is  to  list  the  possible 
states  for  each  node.  The  fourth  step  is  to  establish  the  connection  between  nodes.  These 
connections  can  be  created  by  experts  or  learned  from  the  data.  Finally  probability  and 
conditional  probabilities  need  to  be  elicited  or  learned. 

Once  an  initial  model  is  developed  it  needs  to  be  evaluated  before  it  can  be  used. 
First  the  nodes  need  to  be  examined.  Are  all  the  evidence  and  hypothesis  nodes  present? 
Have  you  minimized  the  number  of  mediating  variable  to  support  calculation  and 
explanation?  Next  examine  the  states  of  each  remaining  node.  Are  the  states  mutually 
exclusive  and  collectively  exhaustive?  Have  you  minimized  the  number  of  states  for 
each  node  looking  for  opportunity  to  merge  states?  Is  it  clear  how  you  would  select  a 
state  for  each  node?  Finally,  examine  the  connections  between  nodes.  Do  the 
connections  correctly  model  conditional  independence?  Have  you  minimized  the  number 
of  multiply  connect  nodes  since  these  significantly  increase  computation  time?  These 
steps  are  summarized  in  Figure  6. 
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HOW  ARE  BNs  DEVELOPED  AND 
EVALUATED? 


Knowledge  Engineer  Works  With  the  Domain  Expert  to 
Elicit  and  Validate  BN  Components  Using  Existing  Tools 
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Figure  6  Developing  and  Evaluated  Bayesian  Networks 


3.0  Project  Description 

For  this  project,  we  implemented  a  combat  ID  Bayesian  network  to  evaluate  two 
investments.  One  investment  was  to  perform  good  sensor  and  database  maintenance 
policies  and  the  other  investment  was  to  integrate  a  new  sensor  but  not  perform  good 
maintenance  on  the  sensors. 

This  project  builds  on  an  earlier  effort  [16],  [17]  comparing  different  probabilistic 
approaches  to  performing  combat  identification.  We  reused  the  deliberately  ambiguous 
scenario  from  the  previous  project.  In  this  operational  scenario  a  ship  is  assigned  as 
battle  group  screen  and  has  confirmed  that  it’s  radar,  communication,  IFF,  ES  equipment 
is  working.  Friendly  aircraft  are  returning  from  a  strike  mission  with  some  aircraft 
reporting  damage.  A  new  aircraft  detected  headed  towards  battle  group  but  the  ship  has 
no  IFF,  no  communication,  and  no  ES  from  the  aircraft.  The  aircraft  is  in  a  return  to 
force  corridor  flying  at  675  knots.  The  ships  Commanding  Officer  has  to  decide  whether 
to  shoot  the  aircraft.  To  help  make  this  decision  a  decision  table  is  created  as  figure  7 
showing  the  consequence  of  each  combination  of  action  and  possible  identity.  A  utility 
for  each  consequence  is  assigned  with  a  utility  of  0  for  shooting  a  friendly  or  commercial 
aircraft  and  a  utility  of  1  for  shooting  a  hostile  aircraft. 

In  order  to  increase  the  robustness  of  this  scenario  the  scenario  is  evaluated  in 
different  environments  described  in  figure  9.  Traditionally,  different  scenarios  would  be 
strung  together  to  form  a  design  reference  mission  for  evaluation.  The  drawback  is  that 
these  scenarios  are  all  from  the  same  future  and  the  utility  may  be  different  futures.  The 
echoes  the  criticism  that  we  are  always  fighting  the  last  war  and  are  surprised.  Scenario 
Analysis  has  been  used  for  many  years  in  business  to  select  make  decisions  that  are 
robust  in  different  futures.  Four  or  five  scenarios  are  drawn  from  different  futures  and 
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policies  are  evaluated  in  each  of  these  futures.  In  the  CID  domain  the  different  futures 
are  distinguished  by  the  ability  to  discriminate  between  hostile  and  friendly  tracks  on  one 
axis  and  hostile  and  neutral  tracks  along  a  perpendicular  axis. 
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Figure  7  Utility  Evaluation 
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Figure  8  Environments 
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4.0  Results 

For  this  project  we  created  and  evaluated  a  Bayesian  Network  using  the  computer 
program  Netica  shown  in  figures  10  and  11. _ 


GOOD  MAINTENANCE  INVESTMENT 


27 


Figure  9  Good  Maintenance  Bayesian  Network 

NEW  SENSOR  INVESTMENT 


28 


Figure  10  New  Sensor  Bayesian  Network 
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4.1  Model  Evaluation 

Model  evaluation  consists  of  four  steps  -  Node  Evaluation,  State  Evaluation, 
Network  Evaluation,  and  Probability  Evaluation. 

The  first  step  was  node  evaluation.  The  nodes  either  are  either  evidence  nodes  or 
hypothesis  nodes.  The  only  mediating  node  is  classification.  It  is  needed  since  it 
represents  the  actual  track 

State  Evaluation  is  next.  Once  node  appropriateness  was  evaluated  the  states  of 
each  node  was  examined.  The  states  of  each  node  were  deemed  to  be  mutually  exclusive 
and  collectively  exhaustive.  Some  of  the  nodes  had  their  states  combined  like  the  IFF 
node  had  the  garbled  and  no  reply  merged  and  the  RSM  and  EW  node  would  have  many 
more  states  in  a  deployed  that  are  collected  in  the  ‘other’  state.  Each  state  also  passes  the 
clarity  test  in  that  it  is  possible  to  unambiguously  assign  a  state  to  each  node. 

The  third  step  in  the  model  evaluation  was  the  network  evaluation  -  do  the  links 
between  the  nodes  make  sense.  The  graph  depicts  two  nodes  that  D-separate  the  graph. 
If  you  know  the  identity  then  you  it  doesn’t  matter  to  evaluating  classification  whether 
you  also  know  the  state  of  the  intel,  IFF  or  comms  nodes.  Similarly  if  you  know  the  state 
of  classification  it  doesn’t  matter  in  evaluating  identity  whether  you  also  know  the  states 
of  the  EW,  RSM,  and  Kinematics  node.  This  network  is  also  singularly  connected  easing 
network  evaluation.  This  network  properly  reflects  causal  relationships  since  the 
classification  causes  particular  identities  and  the  identity  causes  the  IFF  reply  as 
examples 

Finally  an  assessment  was  made  on  how  the  probabilities  would  be  assigned  to 
each  parent  node  -  in  this  case  classification.  Order  of  battle  information  for  each  region 
is  suggested  as  a  way  to  assign  the  probabilities  for  this  network.  This  would  augmented 
by  informed  expert  judgment  for  the  new  sensor  added. 

4.2  Performance  Evaluation 

Table  1  provides  the  contribution  of  the  new  sensor  and  table  2  provides  the 
contribution  good  sensor  maintenance.  For  the  new  sensor  investment  the  performance 
of  the  IFF,  Comms,  Intel,  and  EW  nodes  were  degraded.  The  crystal  ball  environment 
was  the  reference  environment.  For  “Wolves  in  the  Fold”  and  “Foggy  Day”  environment 
the  ES  the  degraded  making  it  more  likely  that  commercial  and  hostile  forces  would  be 
confused.  For  “Dancing  in  the  Dark”  and  “Foggy  Day”  environment  the  IFF  and  Comms 
nodes  were  degraded.  Table  3  compares  the  two  investments.  In  this  scenario,  the 
preferred  investment  is  to  integrate  the  new  sensor. 
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Table  1  New  Sensor  Results 


Environment 

P(H|E) 

P(N|E) 

P(F|E) 

U(S) 

U(DS) 

Wolves  in  the  Fold 

72 

25 

2 

72 

14 

Crystal  Ball 

79 

18 

2 

79 

10 

Dancing  in  the  Dark 

53 

28 

19 

53 

24 

Foggy  Day 

53 

28 

19 

53 

24 

AVERAGE 

64 

25 

11 

64 

18 

Table  Legend: 

P(HIE)  Probability  Object  Is  Hostile  Given  Evidence 

P(NIE)  Probability  Object  Is  Neutral  Given  Evidence 

P(FIE)  Probability  Object  Is  Friendly  Given  Evidence 
U(S)  Utility  of  Shoot  Decision 
U(DS)  Utility  of  Don’t  Shoot  Decision 


Table  2  Good 

Maintenance  Resul 

Its 

Environment 

P(H|E) 

P(N|E) 

P(F|E) 

U(S) 

U(DS) 

Wolves  in  the  Fold 

55 

42 

3 

55 

20 

Crystal  Ball 

55 

42 

3 

5 

20 

Dancing  in  the  Dark 

42 

46 

11 

42 

29 

Foggy  Day 

42 

46 

11 

42 

29 

AVERAGE 

49 

44 

7 

36 

25 

Table  3  Comparison  Of  Investments 


INVESTMENT 

U(S) 

U(DS) 

DIFF 

Good  Maintenance 

36 

25 

12 

New  Sensor 

64 

18 

46 

5.0  Summary 

In  this  paper  the  background  necessary  to  develop  a  Bayesian  Network  to  perform 
CID  was  provide.  Two  CID  investments  were  compared  using  a  Bayesian  network.  For 
the  deliberately  ambiguous  scenario  the  contribution  measured  as  the  difference  in  the 
utility  between  shooting  and  not  shooting  was  compared  and  integrating  a  new  sensor 
was  preferred. 
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